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ABSTRACT 

For evaluation and further development of the 
First-Year Communication Skills Program, a classroom observational 
study was conducted to determine the extent to which teachers 
demonstrated the desired instructional behaviors and to measure pupil 
response rate. A sample of 10 classes was selected for observation by 
two observers during a regularly scheduled lesson on blending (word 
attack) outcomes. Observers were trained in the use of two 
observation scales developed for the study: one for sti mulus- response 
analysis of instructional transactions between teacher and pupils and 
one to measure the patterning and dispersion of pupil responses and 
the extent to which pupils are presented stimuli appropriate to the 
program outcomes. Results were used to revise training objectives and 
develop corresponding materials for the 1969-70 installation. The 
effectiveness of the new training program was then measured, and 
based on results and instruments, new procedures and revised 
classroom monitoring instruments will be developed for use by program 
supervisors in the 1970-71 tryout,. Comparison of trained and 
untrained teachers shows the program effective for pupil response 
rate, relevance of instructional stimuli to lesson outcomes, pupil 
praise, and use of prompts,, Behavioral analysis of instruction 
appears to be useful in identifying training requirements and 
evaluating program effectiveness. (The observation forms and 
description of instructional procedures taught to teachers are 
included.) (JS) 
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BEHAVIORAL ANALYSIS OF FIRST -YEAR COMMUNICATION SKILLS 
TEACHER -ADMINISTERED INSTRUCTION 

Paul E. Resta and Fred C. Niedermeyer 



UJ The First-Year Communication Skills Program developed by the Southwest 

Regional Laboratory (SWRL), specifies a number of procedural rules that 
should be followed by teachers when presenting instructional stimuli and 
confirming pupil responses (see Appendix A) . 

In informal field visits by SWRL staff, it was noted that a number 
of teachers did not appear to comply with the suggested procedures. It 
was also noted that in a number of classes, the response rate of pupils 
during instructional sessions appeared to be extremely low. 

In order to determine the extent to which participating teachers 
demonstrated the desired instructional behaviors and to measure pupil 
response rate, a classroom observational study was conducted by Laboratory 
staff. Teachers' instructional behaviors in the natural environment were 
observed, recorded, and analyzed. The results of the study were then used 
for deriving teacher training objectives and developing classroom moni- 
toring instruments to be used by program supervisors . 



METHOD 



The classroom observation study required that a sample of teacher- 
pupil instructional behaviors be observed and recorded during a regularly 
scheduled blending lesson. The blending lesson was selected for obser- 
vation because it incorporated all of the specified procedural rules for 
instruction. 

In this study, teams of two SWRL staff members observed the teachers 
during the conduct of a program lesson on blending. One of the observers 
recorded the instructional interactions between the teacher and the 
pupils. The other observer recorded the distribution and frequency of 
pupil responses and the content of the teacher's instructional directions 
and questions. 



SAMPLE 




A sample of 10 classes in a suburban middle-class school district 
near Los Angeles was selected for observation. Arrangements were made 
for SWRL staff members to visit each classroom and observe a regularly 
scheduled lesson on the blending (word attack) outcome. (Appendix B 
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and C are samples of the letters used to arrange the observations.) 
Special care was taken to assure that the observers received no prior 
knowledge as to which teachers had high or low performing classes. Only 
one of the teachers included in the sample was not observed in the study. 
Teacher Q (lowest performing class in the district) repeatedly changed 
the scheduled time for the visit. When it became obvious that it would 
not be feasible to visit the class with the teacher's full cooperation, 
the planned visit was finally cancelled. 



INSTRUMENTATION 

Two observation scales were developed for recording the instructional 
interaction between the teacher and her pupils: the Instructional S-R 

Observation Scale and the Response Content and Dispersion Observation 
Form. 



Instructional Stimulus -Response Observation Scale . The first 
observational scale was an adaptation of a scale used to assess the 
instructional behaviors of student tutors.^ The scale was designed to 
provide data for a stimulus -response analysis of the instructional trans- 
actions between the teacher and a group of pupils. Various items of 
information can be extracted from this form, e.g., the proportion of 
total number of responses that were made by individual students, number 
of student responses followed by teacher confirmation, etc. As shown in 
Figure 1, the scale defines specific teacher-pupil behaviors (lettered A 
through P) that occur during a program lesson. The scale provides a 
concise record of the stimulus presentation, pupil response, and con- 
firmation sequence during the instructional session. Each column in the 
grid represents a stimulus -response-confirmation item during the lesson. 
For example, the entire sequence of a teacher presenting a card, the 
child's response, and the teacher's confirmation (or lack of it) would 
be recorded as a single behavioral sequence on the observation scale. 

Each time the teacher presents a different stimulus item during the 
lesson, the observer records the new behavioral sequence in the adjacent 
column by simple numbering in the appropriate grid. 

Response Content and Dispersion Observation Form . A second obser- 
vational scale was developed to provide a measure of the patterning and 
dispersion of pupil responses and the extent to which pupils are presented 
stimuli appropriate to the program outcomes. As shown in Figure 2, this 
observation form consists of three parts. In Part I the observer notes 

the dispersion of responses by keeping a tally of individual responses 
made by all pupils in the class. This is accomplished by drawing circles 
corresponding to the location of each child in the class 'different colors 
are used to distinguish boys from girls). Tallies of the uumber of 

E 



^See Fred C. Niedermeyer, "The Effects of Training on the Instructional 
Behaviors of Student Tutors," SWRL Research Memorandum, March 20, 1969. 
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individual responses made by each pupil are then made in or adjacent to 
the circles. Part II is used to describe the type, sequence, and elapsed 
time of the activities that occur during the lesson. On Part III of this 
instrument the observer notes and tallies the content of the various 
stimuli to which the children are responding during a lesson. It can 
then be determined to what extent the pupils respond to stimuli appropriate 
to the program outcomes. 

Information yielded from this form include: 

1. Number of total responses called for during a lesson. 

2. Number of responses called for which deal with stimuli defined 
for lesson in Outcomes and Materials Guide for that Unit. 

3. Number of responses called for which deal with program stimuli 
other than those defined for lesson in Outcomes and Materials 
Guide (list such stimuli). 

4. Number of responses called for which are directed to 
individually named pupils. 

5. Number of times individual pupils respond incorrectly. 



PROCEDURES 

Initial tryout of the scales and training of the observers took 
place during simulated lessons in which SWRL staff members role-played 
kindergarten classes engaged in blending lessons. One of these simulated 
lessons was video-taped, and this tape was then used to provide further 
practice opportunities for the observers. A reliability index of .86 was 
obtained between observers using the video-taped sequence. 

Following the training sessions, contact letters (see Appendix B) 
were sent to the District Office and then forwarded to each teacher in 
the sample. The teachers then indicated dates that were feasible for 
the classroom observation. Schedules were then established and confirmed 
with the teachers. 

Prior to a scheduled classroom observation, the two SWRL observers 
reported initially to the principal's office for introduction. They 
then were introduced to the teachers and chatted with the teacher to 
reaffirm that SWRL was attempting to gain information about the effec- 
tiveness of the program materials and procedures. At the beginning of 
the instructional session, the observers seated themselves unobtrusively, 
but in a position that provided line of sight with all pupils in the 
class. At the end of the session the observers thanked the teacher for 
her cooperation and indicated how useful the information would be in 
analyzing the effectiveness of the program procedures. 
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After returning from an observation, each of the two observers filed 
a Field Visit Log describing the lesson and anecdotal episodes which were 
unrecordable with either of the observation scales. These logs provide 
additional insights into the teacher-pupil interactions and a sample is 
contained in Appendix D. 



RESULTS 

The data acquired during the observational study is presented in 
Figure 3 and may be summarized as follows: 

1. Instructional Stimuli 

a. Appropriateness of Instructional Stimuli : Items 3, 4, and 

5 deal with the appropriateness of instructional stimuli 
and related pupil responses in light of the specific out- 
comes designated in the First-Year Communication Skills 
Program. The data cited for Item 3 indicate that in five 
of the nine classes less than one-third of the instruc- 
tional stimuli presented during the session were relevant 
to the particular lesson outcomes. Only one teacher in 
the sample (Teacher E) called for a high proportion of 
responses dealing with lesson outcomes (see Appendix C) 
for examples of appropriate stimuli for particular lesson 
outcomes ) . 

2. Pupil Response 

a. Pupil Response Rate . The number of responses made per 
minute in each class (Item 2) ranged from a low of 2.04 
(Teacher 0) to a high of 6.92 (Teacher E) . The average 
response rate for the nine observed classes was 4.16, 
which is only about one response every 15 seconds. The 
low overt response rates are somewhat surprising when one 
considers the fact that the teachers, cognizant of being 
observed, were presumably demonstrating their best instruc- 
tional behaviors. 

b. Proportion of Group vs. Individual Responses . Items 6 and 

8 separate the observed responses into choral and individual 
responses. The Teacher's Manual had indicated that teachers 
should call on individuals twice as often as groups. It can 
be seen from the table that only three of the nine teachers 
met this criterion (Teachers B, C, and 0). One teacher 
(Teacher N) did not elicit a single individual response 
during the instructional session. 
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Figure 3 

Scores on various Teacher-Pupil Interaction Items for Ten 
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Instructional time (minutes) 


16 
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1 




Total response rate (responses 
per minute during lesson) 


3.25 


3.55 2 


.55 


4.72 


6.92 


4.25 


6.08 2 


.04 4 


.06 


4.16 


1 

6.00 


Proportion of responses deal- 
ing with ,-';imuli defined for 
lesson in Outcomes and Mater i- 
als Guide 


.33 
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.26 


.29 
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.29 


.50 
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.68 


.45 


.80 j 


Proportion of responses deal- 
ing with program stimuli not 
defined for the particular 
lesson 
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.41 
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.20 


Proportion of responses deal- 
ing with stimuli or behaviors 
not defined for FYCSP 
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.00 
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.00 
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.17 


.00 ] 


Proportion of responses which 
were choral 


.44 


.22 


.14 


.51 


.39 


.53 
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.30 
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.33 or less ; 


Proportion of choral responses 
after which teacher verbalized 
the correct response 
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.20 
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Proportion of responses which 
were made by individuals 
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.64 


.69 

.44 


No 

Data .50 


-- 


.31 
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c. Proportion of Individual Responses . Figure 4 (Page 9) 
contains the distribution of the number of individual 
responses by pupils in the ten classes. There were 
classes in which every child responded individually at 
least once. The proportion of students in each class not 
called upon for individual responses ranges from a high 
of 1.00 to a low of .05. Averaging across classes, it 

can been seen that 40% of the pupils were never called upon 
to respond individually during the observed instructional 
sessions . 

d. Proportion of Boy vs. Girl Responses , item 14 of Figure 3 
shows that teachers called on boys in proportions that 
closely matched the proportions of boys comprising the 
classes. In the seven classes from which these data v/ere 
obtained, boys comprised 51% of the pupils and were called 
on 50% of the time. 

3. Response Consequence 

a. Teacher Confirmation . After eliciting a choral response, 
it is desirable that the teacher verbalize the correct 
response again for those pupils who may have been incorrect 
and could not distinguish what the rest of the class was 
saying. The scores on Item 7 show that teachers displayed 
this behavior only 16% of the time. For the scores on 
Item 11, it can be seen that only one teacher (Teacher 0) 
confirmed correct pupil responses more than 80%> of the time. 

b. Verbal Reinforcement . During the observed lessons, the 
teachers praised-*- either individual pupils or the class as 
a whole an average of 2.53 times (Item 12). When divided 
by the average instructional time (Item 1), this comes out 
to about one praising statement every six minutes. 

c. Prompting . Item 13 data indicate that all but two of the 
teachers attempted to prompt incorrect pupils. Teacher P 
prompted 20 times. Out of a total of 34 prompts attempted 
by six teachers, only six were successful (i.e., elicited 
the correct response) . 



^A "praising” remark is any positive comment made following a confirmation 
comment, e.g., a single comment such as "Good, Johnny," or "Fine" was 
designated a confirming remark. If such comments were followed by another 
positive remark it would be designated praise, e.g., "Good Johnny, that 
was excellent," or "Fine, you're doing very well." 
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Distribution of number of pupils making individual responses 
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IMPLICATIONS FOR TEACHER TRAINING AND MONITORING OF INSTRUCTION 



Teacher Training . The last column in Figure 3 indicates, where 
appropriate, arbitrary criterion levels for various teacher-pupil 
interaction items. The results of the study indicate that these levels 
were observed for relatively few of the teachers. Thus, several 
objectives were derived for the teacher training program to correspond 
to the various items: 

. Appropriate Instructional Stimulus -Response Sequence (Items 3, 

4, and 5) To distinguish between appropriate and inappropriate 
stimulus materials and response practice for skill development, 
given examples of lessons for each skill. 

. Individual Practice (Items 6 and 8, and Figure 4) To identify 
practice situations conforming to individual practice require- 
ments, given exemplars. 

. Confirmation and Praise (Items 7, 11, and 12) To identify 
appropriate confirmation and praise statements, given several 
examples . 

Correction (Item 10) To identify appropriate procedures for 
dealing with wrong responses and non-responses, given examples. 

. Prompting (Item 13) To distinguish between desirable and 
undesirable prompts, given several examples of each type. 

. Response Frequency (Item 2) To distinguish between instructional 
activities which are likely or unlikely to generate a response 
rate of at least six responses per minute. 

Training materials corresponding to these and other objectives were 
developed for the 1969-1970 installation of the Communication Skills 
Program. 1 The training package for teachers contained a 16mm sound-film, 
various slide-tape components with response sheets, and several self- 
instructional workbooks. Job-aid references (Procedure Cards) were also 
developed for daily teacher use as vjell as instructions and procedures 
for daily individual assessment by teachers. The training program 
underwent two trial-revision cycles during the summer and was administered 
to teachers in the fall who were going to use the program. 

The effectiveness of the training program was measured (1) teacher 
pretest-posttest scores during training, (2) teacher on-the-job behaviors 
during the year, and (3) pupil performance on the program outcomes. 



^See Paul E. Resta "First-Year Communication Skills Installation Training 
Program Specifications," SWRL Research Memorandum, revised, June, 1969. 
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Figure 5 presents a comparison of the mean scores of a random sample of 
10 trained teachers from the same district in comparison to the sample 
of 10 untrained teachers. 

The following items are worth noting in the comparison: 

a. Pupil response rate of trained teachers increased 
almost by half (Item 2). 

b. All instructional stimuli (Items 3, 4, and 5), presented 
by trained teachers, were relevant to Lesson Outcomes. 

c. Slightly more individual responses were called for by 
trained teachers (Item 8) . 

d. Trained teachers had a higher proportion of correct 
individual responses. 

e. A higher proportion of correct individual responses 
were confirmed by trained teachers (Item 11). 

f. Trained teachers praised pupils much more frequently 
than did the untrained teachers . 

g. Trained teachers used approximately one -third as many 
prompts as did untrained teachers. In addition, the 
trained teachers had a much higher proportion of 
successful total number of prompts than did the 
untrained teachers. 

Monitoring of Instruction . Based on the results and instruments 
yielded in the present study, new procedures and instrumentation will 
be developed for use by program supervisors in the 1970-71 Installation 
Tryout. These will be incorporated into a program management system 
that will allow district supervisors to monitor class performance and 
to assist teachers who are not meeting program objectives. 

Revision of Observation Form . The two forms will be revised and 
broken-out into several smaller forms for more convenient application 
in the field. Additional items will be developed (e.g.. Appropriateness 
of Instructional Stimuli to Program Outcome) within existing categories. 
New categories by item will also be established to sample other program- 

relevant instructional behavior. Items are being developed, for example, 
to provide a systematic method of observing and recording the application 
of extrinsic reinforcement by teachers contingent on pupil performance. 
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Figure 5 

Mean Scores on Teacher-Pupil Interactions for 
Samples of 9 Untrained and 10 Trained Teachers 
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.17 


0.16 
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6. 


Proportion of responses which were 
choral 
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0.23 


.39 


.24 


7. 


Proportion of choral responses 
after which teacher verbalized the 












correct response 


.16 


0.13 


.20 


.21 


8. 


Proportion of responses which were 
made by individuals 


.57 


0.23 


.61 


.24 


9. 


Proportion of individual responses 
which were correct 


.82 


0.24 


.88 


.05 


10. 


Proportion of incorrect responses 
for which teacher elicited correct 
response from same pupil 


.56 


0.29 


.59 


.29 


11. 


Proportion of correct individual 
responses which were confirmed by 
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Number of times teacher praised 
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The comparison of the trained vs. untrained teachers suggests that 
the training appears effective for pupil response rate, relevance of 
instructional stimuli to lesson outcomes, pupil praise and use of prompts. 
Much more needs to be done, however, with respect to confirming responses. 



SUMMARY 

Results of this limited study suggests that behavioral analysis of 
instruction can be useful in identifying training requirements and eval- 
uating the effectiveness of program specific teacher-training. Although 
this behavioral analysis was done in a developmental rather than rn 
experimental context, some of the observed dimensions may be relevant for 
other teacher-pupil interaction studies. 
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Appendix A 

GENERAL INSTRUCTIONAL PROCEDURES^ 

Active participation by the children in the program activities is 
essential for successful instruction. Effective procedures for calling 
upon children for oral responses (presentation procedures) and for pro- 
viding information on the correctness of their responses (confirmation 
procedures) are listed below. Except where specific variations are 
noted, these procedures are relevant to all instructional activities in 
the program. 

PRESENTATION PROCEDURES ; 

1. Call on as many different children as possible during each activity 
and keep the pace moving rapidly. A good general rule for the 
various activities is “ Maximize participation and minimize expla- 
nation .” since young children learn well from practice and 
confirmation or correction of their performance. 

2. Call on boys as often as girls. 

3. Call on non-volunteers at least as often as volunteers. 

4. Call on individual children much more frequently than you call on 
groups. Call on groups (e.g., the entire class, all boys, all 
girls, or other sub-groups of the class) primarily when initially 
introducing a new word or sound, or to provide variety after 
calling on several individuals. 

5. Call on the child who is having some difficulty in learning 
Particular words or sounds as often as the children who are having 
no difficulty. 

6. For the flashcard activities, ask the question and give time for 
the children to think of the answer before calling upon a child by 
name. Initially, the class may require some training in this pro- 
cedure so that the children do not give the correct answer before 
a child is called upon. 

7. If a child is unable to read a particular word or sound, help him 
out by providing a prompt that will enable him to respond correctly 
For example, sometimes a child will clearly be unable to read a 
certain item, such as the word *see', when it is presented in 
isolation. However, if he is shown two or more flashcards and 
asked 'Vhich word is *see*?“ he is much more likely to be able to 
answer correctly. 



^From Firi’t-Year Coiranunication Skills Teacher’s Manual. 1968-69. 



















CONFIRMATION PROCEDURES; 



1. Each time a child or group reads something correctly or gives a 
correct answer to a question, make a brief positive remark (e.g., 
"Good"; "Tl^at's right class"; "Very good, Johnny"; etc.) to let' 
them know that they have done well. 

2. When a child gives an Incorrect answer to a question, do not tell 
him directly that bis answer was wrong or Incorrect. Instead, tell 
him the correct answer or give him a prompt so that he can answer 
correctly. Then repeat the question and have him answer It correctly. 
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Appendix B 

Sample Contact Letter 
for Teacher 



Dear : 

SWRL is presently in the process of revising a number of First Year Commu- 
nication Skills Program materials and procedures. 

To help us make sound program-revision decisions, we are particularly 
interested in assessing the adequacy of the current program procedures and 
the implications for teacher-pupil interactions. 

We have selected your class as one of a sample that will be representative 
of most classes in the program. If agreeable, we would like to have two 
SWRL personnel observe a teacher -led (not tape) program lesson on blending. 
The personnel would sit in the back of the classroom and would only remain 
for the period of the blending lesson. We would prefer that you plan your 
lesson for a unit outcome on blending in the usual way. The visit by the 
SWRL staff can be arranged to coincide with the normal time for the lesson. 

On the enclosed reply letter, please note the blending outcomes which you 
plan to present during the two-week period from April 21 to May 2, and the 
daily time when your lessons are usually presented. We will plan our 
observation to coincide with your normal lesson time, and will inform you 
of the scheduled observation date and time by April 18. Please return 
your reply to SWRL by Monday April 14. 

We appreciate your continued help and cooperation in the implementation 
and evaluation of the SWRL Communications Skills Program. 

Sincerely , 



Robert W. O'Hare, Element Head 
Educational Resource Service 

RO'H:mp 

Enclosure 
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Appendix C 
Sample 

Teacher Reply Letter 



Dear SWRL, 

I agree to have SWRL observers during a teacher presentation of a blending 
lesson. 



During the two-week period from April 21 to May 2, I anticipate presenting 
lessons on blending as checked and dated below. 



Unit 7, Outcome 2 

(3 for taped) 

Unit 7, Outcome 5 

(6 for taped) 

Unit 7, Outcome 7 

(8 for taped) 

Unit 7, Outcome 9 

(10 for taped) 

Unit 8, Outcome 3 

(4 for taped) 

Unit 8, Outcome 9 

(10 for taped) 

Unit 9, Outcome 14 



land, .let, lad 
week, need, feed 
sheet, shell 
set, met, wet, net 
hut, nut, shut 
hat, hit, had 
men, then, hen 



Check if it will 
be used during 
this time -period 



Give anticipated 
date for presen- 
tation, if possible 



My class lessons are usually presented each day at the following time: 



I understand you will inform me of the scheduled observation date and time 
by April 18. 



Teacher 

School 
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Appendix ‘D 

FIELD VISIT LOG 



h.- May 2, 1969 

To; Or. R. Boktr 

Dr- F. Ratta 

-■ , ■ inttaliation Fila 

Product Intagration Fila 

(Othar) 

Fiald Raprasantativa; 



School Viiitad; 



TcMhar Viiitad; 

Teacher A 



The classroom was very orderly and quiet. The children were paying 
attention to what Teacher A said and to what the other children said 
and did. She was positive in her attitude, and discouraged the children 
from criticizing each other. 

The lesson began at 9:10 when she placed cards with "en" on a card 
chart and passed out cards with initial sounds to volunteers. They 
brought the cards to the chart, made a word, and turned to the class 
to say the blend ("m-3n, men," etc.). One little girl was saying 
"p" instead of "b," so she sent her to the side chalk board where 
the alphabet was displayed. The girl came back knowing that the n;ame 
of the letter was "b," but still saying the "p" sound. Teacher A 
said the "p" sound and had the girl repeat it until she said it 
correctly ("b" is the initial sound for the next unit). 

At 9:15, she wrote "hen," "men," and "when" on the board and called 
for choral blending response. She then wrote "end" on the board, put 
different initial sounds in front of it, and asked them to chorally 
blend the sounds. They seemed to do well on this. She then changed 
the ending to "ent" and did the same thing. 

At 9:22, she put "en" on the board and chose volunteers to come up 
and "make it into a big word." After writing the word, the child 
turned to the class and said the blend ("h-en, hen," etc.). This 
lasted until 9:26, at which time the lesson was over. 

As can be seen from the observation sheet, half of the class (15 
children) never got called on (although they were all paying attention 
to what was being said). Because it took so much time for them to 
write the words on the board, there were not as many total responses 
as we have seen in some other classes (58 compared to the approximately 
100 in two other classes). Of the 58 total responses recorded, 22 were 
individual responses made by 15 children. 

Because she used "end" and "ent" endings when she was writing words 
on the board for them to blend, and because the children also used 
these endings when they were making words with "en," I would say 
that a total of from 7-8 minutes was spent on the "en" ending, which 
is the ending for this unit. Twice when she was asking for "en" 
words, a child suggested a word without "en" in it. She said, "We'rci 
only working with "en" words today." 



ERIC 






















